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Super-resolved far-field microscopy has emerged as a powerful tool for investigating the structure 
of objects with resolution well below the diffraction limit of light m- Nearly all super-resolution 
imaging techniques reported to date rely on real energy states of probe molecules to circumvent 
the diffraction limit, preventing super-resolved imaging of contrast mechanisms that occur via vir¬ 
tual energy states such as harmonic generation (HG). Here we report a super-resolution technique 
based on SPatlal Frequency modulated Imaging (SPIFI) that permits super-resolved nonlin¬ 
ear microscopy with any contrast mechanism, and with single-pixel detection. We show multimodal 
super-resolved images with two-photon excited fluorescence (TPEF) and second-harmonic gener¬ 
ation (SHG) from biological and inorganic media. Multiphoton SPIFI (MP-SPIFI) provides 
spatial resolution up to 2rj below the diffraction limit, where rj is the highest power of the nonlinear 
intensity response. MP-SPIFI has the potential to not only provide enhanced resolution in optically 
thin media, but shows promise for providing super-resolved imaging at depth in scattering media - 
opening the possibility of in vivo super-resolved imaging. 


I. INTRODUCTION 

Of the numerous far-field image formation techniques, multiphoton microscopy (MPM) has proved particularly 
valuable in many biological studies, providing multimodal images with robustness to optical scattering |S] , and label- 
free images of complex biological structures [QfflT] and inorganic harmonic probes nails]. Scattering robustness 
is achieved twofold with nonlinear excitation. First, inherent optical sectioning provided by the nonlinear intensity 
dependence m allows for collection of signal light with a single-pixel detector. Second, the use of excitation wave¬ 
lengths in the near infrared (NIR) |15l ITT)] enhance penetration depth of the ballistic photons responsible for nonlinear 
contrast generation m- Due to the diffraction limit of light, these longer excitation wavelengths yield lower spatial 
resolution than wavelengths in the visible spectral region, such as those used for confocal fluorescence microscopy. 

The most successful super-resolution imaging techniques reported to date have exploited the real energy states of 
probe molecules to generate image contrast. These real energy states provide a means to manipulate how contrast is 
generated, thereby reducing the effective size of the point-spread function (PSF) which sets image resolution. Some 
examples include photoactivated localization microscopy (PALM) |2j|T8|, stimulated emission depletion (STED) mi¬ 
croscopy inmnilinij ground state depletion (GSD) microscopy |21l[2?| . saturation of transient absorption in electronic 
states |13|, and saturation of scattering due to surface plasmon resonance |241125] . 

Gonversely, the instantaneous response of the contrast medium in coherent nonlinear scattering results in no net 
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energy transfer. In this case, the photokinetics are often described via virtual energy states. The application of most 
super-resolution imaging methods to virtual energy states is not possible. Despite this, examples of HG microscopy 
with sub-diffraction-limited resolution have been reported Eni[27]. While these methods succeeded in collecting HG 
images with sub-diffraction limited resolution, both are unlikely to be robust to imaging at depth in tissue due to 
effects such as birefringence and circular dichroism. 

Until now, a technique that provides super-resolved imaging of any contrast mechanism in complex media has 
not yet emerged. Here we report the use of MP-SPIFI for super-resolved imaging of SHG and TPEF. Gombining 
the scattering robustness offered by illumination wavelengths in the NIR with single-pixel detection of signal light, 
MP-SPIFI has the potential to provide super-resolved images at depth in scattering media. 


II. PHYSICAL BASIS OF RESOLUTION ENHANCEMENT 


The underlying principle of MP-SPIFI is to illuminate the specimen with a spatio-temporally varying intensity 
pattern, /in(r, t), that contains a discrete set of well-separated spatial frequency components. The fundamental lateral 
(x) spatial frequency of the illumination intensity in the object region, fx, 2 , is varied by modulating an incident laser 
beam with an amplitude grating whose fundamental spatial frequency, fx,i, varies with time. Signal light, /3(r,t), 
generated by the interaction of probe molecules in the specimen with the illuminating intensity, is collected on a 
single-element photodetector such as a photomultiplier tube (PMT). The signal from the photodetector is expressed 
mathematically as: 


5(t) = (/3(r,% = ([/in(r,t)f C(r,t)), (1) 

where C{r,t) is the spatiotemporal distribution of probe molecules, including parameters such as cross-section, quan¬ 
tum efficiency, etc., and {■)^ denotes integration over all space. 

Physically, Eq. 0 represents a projection of the spatial frequencies present in the illumination intensity onto the 
object, thereby encoding the amplitude of these spatial frequency components that are present in C{r,t). Since 
fx .2 is varied through the entire span of frequency support provided by the illumination system, S{t) encodes a 
one-dimensional (ID) representation of the specimen in the lateral spatial frequency domain. 

A schematic of the MP-SPIFI microscope in the lateral-axial plane is shown in Fig. 0- Schematics showing the 
microscope in both the lateral-axial and the vertical-axial planes, (x, z) and (y, z) respectively, are shown in Fig. ^ 
In the mask plane, a spinning disk with rotational frequency Vr imparts a time-dependent lateral spatial frequency, 
fx,iit), that varies linearly with time, causing the vertically-focused pulse train incident on the mask to diffract into 
several beams with varying propagation angles, 9ij(t), where j is the diffracted order. 

In the object plane, conjugate to the mask plane, the diffracted orders interfere to form a spatial intensity pattern 
containing a discrete set of spatial frequencies that vary linearly with scan time. The spatial frequencies contained in 
the illumination intensity at any time t are harmonics of the fundamental spatial frequency imparted by the mask. 
In the case drawn here, where j € { — 1, 0,1}, the spatial frequencies in the intensity pattern are {0, fx, 2 (t), ‘^fx, 2 it)}, 
where fx, 2 it) = M fx,i{t), rii and 712 are the indices of refraction in the mask and object planes respectively, 

and M is the magnification of the imaging system, defined as the ratio of the geometric focal lengths of the tube lens 
and the objective lens, M = Ft/Fo- 

The fundamental spatial frequency in the object region, fx, 2 it)j appears in the intensity due to interference of 
the j = {0,1} and j = {0,-1} beams, while the second-harmonic of the spatial frequency, 2 fx^ 2 {t), appears due 
to the interference of the j = ±1 beams. In a linear-excitation SPIFI microscope, S{t) encodes two images of the 
specimen j4]: one with lateral spatial frequency support limited by the cutoff frequency of the objective lens, defined 

fx,c = NA/A, and a second image with spatial frequency support of 2/^, g. 

When the signal light /3{r,t) depends nonlinearly on the illumination intensity, the frequency support range is 
expanded further by additional harmonics of fx, 2 {t) that appear due to the spatial nonlinearity. For example, nonlin¬ 
earities in fluorescence intensity arising from saturation of linear electronic absorption, similar to saturated structured 
illumination microscopy (SSIM) [T], could be used to project spatial frequencies beyond those supported by the ob¬ 
jective lens. In this case the signal light can be represented as /3{r,t) = f [Ii\\{r,t)]C{r,t), and the functional form 
/[•] can be expanded in a Taylor series to find the relative weights of higher-order spatial frequency components |28j . 

Similarly, nonlinearities arising from multiphoton contrast lead to additional spatial frequency support. Here we 
consider nonlinearities of integer-order 77 , such that P{r,t) = C{r,t). Gonsequently, the temporal signal 

collected from the PMT encodes a set of 2ri images with spatial frequency support up to 277 -times the cutoff of 
the objective lens. This can be easily shown by examining the spatio-temporal illumination intensity in the lateral 
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FIG. 1: Physical principles of super-resolved imaging with MP-SPIFI. (a) Schematic representation of the imaging 
system, (b) A decomposition of the nonlinear illumination intensity into constituent harmonics of spatial frequency. 
DC spatial frequencies have been omitted for simplicity. T = l/i/j. is the period of mask rotation. 
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FIG. 2: SHG-SPIFI signal measured from a single BaTiOa particle. Each SHG-SPIFI image, Sq+{i^t), is centered 
about a harmonic of the carrier frequency, allowing for separation of each image. Here we plot the mean signal from 

1000 measurements to enhance the signal-to-noise ratio. 


dimension (a;) with scan time (Supplementary Information): 


2r) 2?7 

[/m(a:,t)]"' = '^bq{t) cos [2tt q f^^2(t) x + qi^ct] = ( 2 ) 

g=0 9=0 

where I'c = M Ak Vr Xc is a carrier frequency that arises from the non-zero modulation frequency provided by 

the mask on the optic axis of the microscope (Fig. Ak sets the density of the modulation mask |3]. 

Importantly, the carrier frequency allows for immediate separation of all the MP-SPIFI images, circumventing the 
need to acquire multiple images to recover spatial frequency information beyond the cutoff of the imaging system. 
With appropriate selection of the rotational frequency of the mask, all images may be recovered in a single measure¬ 
ment. Figure^ shows the nonlinear illumination intensity in the lateral dimension versus scan time, as well as the 
corresponding decomposition of the intensity into harmonics of fx, 2 {t)i where the carrier frequency for a given lateral 
position is seen to increase harmonically for each order. This is also clear from the experimental TPEF-SPIFI data 
displayed in Fig. 

Inserting the nonlinear illumination intensity into Eq. Q, we find that the MP-SPIFI signal is a sum of 2 77 images. 
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FIG. 3: MP-SPIFI provides increased lateral spatial frequency support. Experimental ID MTFs from (a) TPEF, 
measured from 100-nm FNDs, and (b) SHG, measured from 200-nm BaTiOa particles. The gray shaded regions 
represent the bounds of frequency support for linearly-excited imaging with A=1065 nm and NA=0.8. 


each with lateral spatial frequency support up to q ^-nd carrier frequency q 


2 ri 277 

bq{t) (cos [27r q fx, 2 {t) x + qv^t] C{x, t))^ = (3) 

9=0 9=0 

For each rotation of the modulation mask, the voltage vs. scan time measurement of /3(r,t) from the photodetector 
encodes 2r] line images, all with varying spatial frequency support. The images are easily separated from one another 
by a Fourier transform of the photodetector signal, as the carrier frequency associated with each MP-SPIFI order 
ensures that the images do not overlap in the modulation frequency spectrum with proper selection of Vr- 

Utilizing the Abbe definition we can write the spatial resolution of the MP-SPIFI order as: Sxq = A/(2NAg) = 
l/{2q fx^c), where NA^ is the effective NA of the g^^-order image. Since the maximal spatial frequency support is 
achieved when q = 2r], the best possible spatial resolution in MP-SPIFI is da^min = 5xi/{2r]) when j G {—1,0,1}. 


III. RESULTS 

To demonstrate that MP-SPIFI provides images with frequency support beyond the diffraction limit for contrast 
mechanisms with both real and virtual energy states, we used MP-SPIFI to image TPEF from 100-nm-diameter 
fluorescent nanodiamonds (FNDs), and SHG from 200-nm-diameter barium-titanate oxide (BaTiOa) crystals. The 
objects were illuminated with femtosecond laser pulses centered at a wavelength of 1065 nm and focused by a 0.8 NA 
objective lens. By isolating and demodulating each MP-SPIFI order, the ID optical transfer function (OTF) was 
recovered for each image. 

Figure l^shows the modulation transfer function (MTF) for each SPIFI order, \Sq{fx, 2 )\, where we have used the 
fact that jx, 2 {'t) = {n 2 /ni) M Ak Vr t to scale the temporal axis to the lateral spatial frequency domain. It is clear that 
lateral spatial frequency information beyond the cutoff frequency of the objective lens is contained in all MP-SPIFI 
orders for g > 1. 

We demonstrated super-resolved multiphoton imaging of biological media by imaging TPEF from polymerized 
tubulin fibers (i.e. microtubules) tagged with with Alexa 546 in a mitotic HeLa cell, and SHG from collagen fibers 
in fixed rabbit tendon (Fig. |^. One-dimensional MP-SPIFI images were collected in the horizontal dimension as 
the specimen was scanned vertically to form a two-dimensional (2D) image. Resolution enhancement occurs only in 
the horizontal dimension. Lateral tomography has been used for 2D image capture with linear SPIFI, and could be 
applied to MP-SPIFI for super-resolved 2D imaging [ 3 . 

To demonstrate simultaneous multimodal super-resolved imaging, we imaged photoluminescence (PL) and SHG 







5 



FIG. 4: TPEF and SHG images collected from biological media with MP-SPIFI. (a) First- and (b) second-order 
MP-SPIFI images of TPFF from a mitotic HeLa cell immunostained with primary antibodies against alpha tubulin 
and secondary antibodies tagged with Alexa 546. Scale bar: 3 /iin. (c)-(f) First through fourth order images of SHG 
from fixed, 16-/rm-thick rabbit tendon. Ringing in the third-order image (e) was attributed to modulations in the 
corresponding MTF (cf. Fig.|^. Scale bar: 10 /rm. All images were collected at 0.8 NA with laser pulses centered at 
1065 nm. TPFF was collected in the epi direction and SHG was collected in the forward-scattered direction. 



FIG. 5: Multimodal super-resolved images of SHG and PL from GdTe solar cells, (a)-(d) First through fourth 
order SHG images, (e)-(h) Gorresponding PL images, (i)-(l) False color merged images. PL and SHG were 
measured in the epi-direction simultaneously. Scale bar: 10 /rm. 
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from CdTe solar cells (Fig. [^. Due to the opacity of the CdTe cells, both contrast mechanisms were collected 
simultaneously in the epi-direction. Figures [^-d display the first- through fourth-order SHG-SPIFI images, and the 
corresponding PL-SPIFI images are shown in Fig. -h. Comparison of the images for each MP-SPIFI order makes 
it clear that the spatial resolution is enhanced in both SHG and PL simultaneously, although the resolution appears 
to be different for these two contrast modes beyond the second-order data. The reason for this discrepancy is unclear 
at this time, though we note that the relatively long lifetime of the PL contrast mechanism, as well as diffusion of 
excited carriers may be responsible for these differences. 


IV. DISCUSSION 

Although the spatial frequency support attainable for TPEF and SHG imaging with MP-SPIFI is four times that 
provided by the objective lens, not all images shown in this work reach this theoretical limit. For example, TPEF- 
SPIFI images obtained from tubulin in HeLa cells were limited to a resolution enhancement of 2 x. The loss of spatial 
frequency information in higher-order MP-SPIFI images is attributed to the reduction in signal-to-noise ratio (SNR) 
with increasing MP-SPIFI order. This loss of SNR has two primary causes. 

First, the noise floor in the modulation frequency (spatial) domain is determined by the shot noise from the time- 
averaged total light intensity measured on the photodetector, (/3(r,t))j.. Gonversely, each MP-SPIFI order represents 
only a portion of this total light intensity. Since the shot-noise-limited noise floor scales as (/3(r,f))j,, the SNR of 
higher order MP-SPIFI images scales as {j 3 q{r,t)) /(/3(r,f))^,. This has also been observed in fluorescent imaging 
using radiofrequency-tagged emission (FIRE) |2H] . another imaging method that uses spatial frequency-modulated 
illumination to form images, and the spatial analog has been analyzed in detail for off-axis SHG holography j30| . 
Images of TPEF collected from HeLa cells truncated at second-order due to a large noise floor that restricted our 
ability to collect higher-order images. On the other hand, images of SHG from rabbit tendon had a significantly larger 
SNR due to increasing the power of the illumination laser by ^50%, making the fourth-order signal detectible above 
the shot noise floor. 

A possible route to enhance the SNR in higher-order MP-SPIFI images is the use of a phase modulator in place of the 
amplitude modulating disk, thereby eliminating the undiffracted (j = 0) beam in the object region and significantly 
reducing the shot noise floor. 

Secondly, vignetting of the diffracted beams (|jj > 0) in the pupil plane of the objective lens causes a reduction in 
both transmitted energy and spatial frequency support in the vertical (y) dimension. We investigated the effect of 
the circular objective pupil for each MP-SPIFI order using numerical simulations. First, the throughput energy of 
each diffracted order with scan time was computed with a simple overlap integral (Fig. The loss in energy of 
the higher-order diffracted beams, particularly the j = ±1 beams, causes a reduction in the modulation depth of the 
illumination intensity pattern, reducing the amplitude of each MP-SPIFI order with respect to the background DG 
signal. 

We also investigated the effect of the circular pupil aperture on the spatial frequency support of the diffracted 
beams in the vertical dimension with a numeric calculation of the electric fields in the object region with a vectorial 
angular spectrum formalism |311 I32| (Supplementary Information). Gomparison of the spatial frequency support 
obtained from the numeric model confirms that the spatial frequency support is reduced by vignetting from the 
objective lens (Fig. Despite this effect, MP-SPIFI shows promise for attaining multimodal super-resolved images 
of nonlinearly-excited contrast mechanisms. 

The ability to resolve fine spatial features in an image depends on both the shape and the extent of the spatial 
frequency support of the OTF. To evaluate the relative imaging performance of MP-SPIFI compared to conventional 
LSM techniques, the OTFs of these methods were computed using the vector focusing numerical model for two-photon 
SPIFI (TP-SPIFI), and two- and three-photon LSM. Intensities were computed for an illumination wavelength of 
1065 nm, 0.8 NA, and linearly-polarized light aligned with the vertical (y) direction. For TP-SPIFI, the widths of 
the intensity distributions in the pupil plane were modeled after the MP-SPIFI system reported here (Supplementary 
Information). For the multiphoton LSM calculation, the beam size in the vertical dimension of the TP-SPIFI micro¬ 
scope was used as the radius of the circularly-symmetric Gaussian distribution. The lateral spatial frequency support 
was then computed from the illumination intensity for each imaging type. The resulting spatial frequency support is 
shown in Fig.[^ where the shaded grey regions indicate the spatial frequency support of each LSM mode, while solid 
lines indicate the spatial frequency support for each TP-SPIFI order. 

What is clear in Fig. is that MP-SPIFI imaging collects significantly more information content at high spatial 
frequencies than multiphoton LSM, thus providing the ability to extract significantly more spatial information. As 
the scan time progresses in TP-SPIFI, the focusing of the transmitted diffracted orders is perturbed by the circular 
objective pupil, which shapes the spatial frequency support of the OTF. The MTFs associated with TP-SPIFI extend 
to greater spatial frequency extent for the second-order nonlinear interaction than either two- or three-photon LSM. 
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FIG. 6: Comparison of the amplitude of the transfer function in multiphoton laser scanning microscopy and 
two-photon SPIFI. The shaded grey regions indicate the frequency support for two- and three-photon laser scanning 
microscopy. Solid lines indicate the frequency support in TP-SPIFI where only first-order diffraction from the 
modulation mask was included. All OTFs were computed with the Debye integral using A = 1065 nm and 0.8 NA. 


Moreover, the amplitude of the high spatial frequency content for TP-SPIFI is significantly higher than for multiphoton 
LSM. The result is that TP-SPIFI captures significantly more fine spatial information about an image that multiphoton 
LSM. 

In summary, we have described a new approach to super-resolved imaging in multiphoton excited microscopy that 
is capable of measuring spatial frequency information beyond the diffraction limit for both multiphoton fluorescence 
and coherent scattering. The combination of excitation wavelengths in the NIR and single-pixel detection should 
allow MP-SPIFI to form images with enhanced spatial resolution at penetration depths well beyond those attainable 
with linearly-excited fluorescence. MP-SPIFI enables super-resolved imaging of new contrast mechanisms, which may 
lead to new insights in both biological and material science studies. 


V. MATERIALS AND METHODS 

Femtosecond laser source. The microscope system is illuminated by pulses from a nonlinear fiber amplifier centered 
at 1065 nm, built in-house |85| . The pulses are generated by an all-normal dispersion fiber oscillator using Yb-doped 
gain fiber [ST], passed through a tunable narrow spectral bandpass filter, and then amplified in Yb-doped fiber. The 
tunable filter varies the launch condition into the nonlinear amplifier, resulting in varying spectral broadening via self- 
phase-modulation. At our typical operating point this resulted in a bandwidth of 28 nm (full-width at half-maximum; 
FWHM). The pulses are temporally compressed in a folded Martinez compressor, resulting in near transform-limited 
durations of approximately 150 fs at an average power ranging from 900 mW to 1.3 W at a repetition rate of 52.5 MHz. 

Optical setup. The beam from the laser is collimated and expanded with a 4:1 Keplerian telescope composed of two 
achromatic lenses (ThorLabs, AC254-100-C-ML and AC254-400-C-ML) obtain a beam size of 8.85 mm (full-width at 
half-maximum of the intensity). The beam is brought to a line focus on the modulation mask with an achromatic 
cylindrical lens (ThorLabs, ACY-254-150-B) oriented such that the beam is focused in the vertical (y) direction only. 

The mask plane is re-imaged to the object plane in two stages, first with a 1:1 image relay system composed of 
two achromatic lenses (ThorLabs AC254-100-C-ML) in a 4-/ configuration, and again with an image relay system 
consisting of a tube lens (ThorLabs AC254-150-C-ML) and an infinity-corrected objective lens. Data presented in 
this work were collected with a 0.8 NA objective lens (Zeiss, N-Achroplan 50x/0.8 NA Pol), such that the mask is 
imaged to the object region with de-magnification of 46.5. 

Modulation mask. The modulation mask consists of a pattern defined in polar coordinates by the expression 
m{r,(p) = 1/2-I-1/2 sgn[cos(Afc r (/;)]. The parameter AA: defines the density of features on the mask. The modulation 
mask used in this work has a density parameter Ak = 70/mm. The disk was mounted on a motor (Faulhaber, 
2057S012BK1155) with a custom chuck. The rotation speed of the disk was controlled by an external speed controller 
(Faulhaber, MCBL3006). 

While the modulation mask imparts transverse spatial frequencies to the line focus in both the x and y dimensions, 
only the spatial frequency in the x direction varies with the rotation angle of the mask. The incident beam is 
diffracted into multiple orders denoted by the integer j. Since the modulation mask is composed of a binary amplitude 
modulation pattern, the set of diffracted orders for an ideal mask is composed of only the undiffracted beam (j = 0) 
and odd diffracted orders. 

Signal collection and digitization. All images displayed in this work were collected in the epi-direction, which was 








achieved by use of a long-pass dichroic beamsplitter placed between the tube lens and the objective lens (Semrock, 
FF875-Di01). 

Fluorescent light from the HeLa specimen was collected in the epi-direction through a short-pass filter (Semrock, 
FF720/SP) and a 500 nm long-pass filter (Chroma, ET5001p). 

Sub-diffraction-limited objects. Samples used for point spread function measurements were 200 nm BaTiOa parti¬ 
cles (1148DY, Nanostructured & Amorphous Materials, Houston TX) for SHG, and 100 nm fluorescent nanodiamonds 
(ND-400NV-100nm-10mL, Adamas Nanotechnologies, Raleigh NC) for TPEF. In both cases the insoluble nanoparti¬ 
cles, suspended in water, were diluted, placed in an ultrasonic bath to break up aggregated particles, and then drop 
cast onto a microscope slide, where the water evaporated away before being cover-slipped. 

Immunofluorescence. HeLa cells were seeded onto glass coverslips and fixed when they reached ^70% confluency. 
Cells were rinsed rapidly with 37 C IX PHEM buffer (60 mM PIPES, 25 mM HEPES, 10 mM EGTA, 4 mM MgS 04 , 
pH 7.0) followed by lysis at 37 C for 5 min in freshly prepared lysis buffer (IX PHEM + 0.5% Triton-X-100). Cells 
were then fixed on the bench top for 3 min using ice cold methanol (95% methanol + 5 mM EGTA) followed by an 
additional 20 min methanol fixation at 20 C. At room temperature, cells were then rehydrated with IX PHEM then 
rinsed 3x5 min in PHEM-T (IX PHEM + 0.1% Triton-X-100) and blocked in 10% boiled donkey serum (BDS) in 
PHEM for 1 hr at room temperature. Microtubules were labeled with anti-alpha tubulin primary antibodies (Sigma- 
Aldrich, St. Louis, MO) diluted in 5% BDS for 12 hr at 4 C. Following primary antibody incubation, cells were rinsed 
3x5 min in PHEM-T and then incubated for 45 min at room temperature with secondary antibodies conjugated to 
Alexa 546 (Life Technologies). Cells were then rinsed 3x5 min in PHEM-T, incubated in a solution of 2 ng/mL 4,6- 
diamidino-2-phenylindole (DAPI) diluted in PHEM, rinsed again (3x5 minutes), and then mounted onto microscope 
slides in an anti-fade solution containing 90% glycerol and 0.5% N-propyl gallate. Coverslips were sealed with nail 
polish to affix them to the slides. 

CdTe crystals. A polycrystalline, large grained CdTe sample was grown by close spaced sublimation (CSS), treated 
with CdCl 2 , and polished. This has the effect of “sealing” small grains (^2 /rm) after growth such that average grain 
size becomes larger. The sample was polished using an ion beam technique, basically like sputtering yet instead of 
adsorption, it simply erodes irregularities in structure away such that the surface is smooth. 

Data analysis. All data analysis was performed in MATLAB (The MathWords, Natick, NJ). Each ID line image 
was reconstructed by computing the fast Fourier transform (FFT) of the measured temporal data S{t) to obtain the 
MP-SPIFI images in the modulation frequency domain, i.e., <§(t't). Since S{t) is a real signal, copies of each image 
are located at both positive and negative carrier frequencies in the modulation frequency domain. Each MP-SPIFI 
order was recovered by numerically filtering at the positive carrier frequency. The filtered data, Sq{vt), was inverse 
transformed to obtain Sq+{t). The complex temporal trace was then demodulated by the relative carrier frequency, 
q Vc, and the temporal axis was calibrated according to the relation fx, 2 it) = q (n^lni) M Ak t to obtain Sq-^.{fx, 2 )■ 
Images were obtained by again taking the FFT of the data. 

Data was calibrated by measuring light transmitted through a 2 /rm slit in the object plane as a function of lateral 
position. The centroid of the measured distribution was computed as a function of lateral position. This allowed 
the quantity describing the relationship between lateral position and carrier frequency, k = (n 2 /ni)M Akv^ to be 
recovered empirically. 

Imperfections in disk mounting lead to an additional phase accumulated in S{t) as a function of mask rotation. 
The process used to remove disk aberration phase has been discussed at length in a previous study |35| 
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SUPPLEMENTARY INFORMATION 
VI. THEORETICAL ANALYSIS OF SUPER-RESOLVED MP-SPIFI 

Here we derive the mathematical framework for MP-SPIFI and show how spatial frequency information that lies 
outside the cutoff frequency of the objective lens is encoded into a temporal measurement of signal light with a 
single-element photodetector. We begin by deriving a general expression for the illumination intensity, and hence the 
signal, then consider the case where only first-order diffraction from the modulation mask contributes to the signal. 

Throughout the analytic derivation, we consider plane wave illumination of the modulation mask for simplicity. We 
have rigorously analyzed plane wave illumination in a previous report and adopt much of the notation here |36| . 


A. General expressions 


As discussed in the manuscript, images are formed in the SPIFI microscope by projecting illumination intensity 
patterns into the specimen and collecting signal light emitted from the object on a single-element detector. The 
illumination intensity is formed by the spatial interference of multiple light sheets propagating at varied angles with 
respect to the optic axis. The density (spatial frequency) of the interference fringes in the illumination intensity is 
precisely controlled by use of a circular glass disk on which is printed an amplitude modulation mask. A spatially- 
coherent illumination beam is brought to a line focus on the modulation mask, and the mask plane is then imaged to 
the object region with a tube lens and an objective lens (cf. Fig. and Fig. #)• 

During a single rotation of the disk, the fundamental spatial frequency of the interference fringes is varied through 
the entire range of spatial frequencies supported by the image relay system. As the disk rotates, the signal light, 
which is generated by the interaction of the illumination intensity and the distribution of molecules in the object, 
is collected on a single element detector. The temporal signal measured from the photodetector is expressed by 
integrating the contrast signal over all space. The contrast signal, /3(r, t), is the product of the illumination intensity 
pattern raised to the power of the nonlinearity, 77 , and the contrast distribution function, C(r, t), which describes how 
the contrast-producing molecules within the object are spatially distributed as a function of time: 

00 

S{t)= I d^r/3(r,t) = ([/in(r,t)]’^ C(r,t))^ (4) 

— 00 


We note that the nonlinear dependence on the illumination intensity need not be integer valued, as is the case in 
harmonic generation (HG) microscopy, for example. If the nonlinearity is not a simple power dependence, one could 
express the contrast light as I3{r,t) = f [/iii(r, t)] C(r,t), where /[•] is a function describing, for example, saturation 
of electronic absorption in target molecules |28| . The functional dependence of the illumination intensity can be 
expanded in a Taylor series, and the highest power of the Taylor series that contributes to the signal will determine 
the cutoff spatial frequencies in super-resolved MP-SPIFI. To simplify the analysis that follows, we only consider the 
case where the contrast intensity depends on an integer power of the illumination intensity. 

The signal measured from the photodetector, S(t), represents the projection of the illumination intensity onto the 
spatial frequency content of the object. We denote the lateral spatial frequency experienced by the illuminating light 
sheet in the mask plane as while fx, 2 {t) is the spatial frequency in the object region. At a given scan time, tg, 

the principle spatial frequency projected onto the object is fx, 2 {ts), and correspondingly the signal amplitude of the 
signal S{ts) encodes the relative amplitude of fx,2{ts) in the object. The whole temporal signal S{t) represents the 
product of the spatial frequency distribution of the object and the spatial frequencies projected into the specimen, 
thus encoding a one-dimensional (ID) image of the object in the spatial frequency domain. 

Here we wish to derive the form of the signal from the photodetector, S{t). Since the illumination beam is spatially 
coherent, the modulation mask causes the incident light beam to be diffracted into multiple beams propagating 
at differing angles with respect to the optic axis. Each diffracted beam has a corresponding wavevector kj,i(t) = 
kx,j,i{i) X -l- kyj^i(t) y -|- kzj^i{t) z. Directly behind the mask, we may write the spatial portion of the electric field for 
the diffracted order j as: 


= o,- e 


i fc; 


,j,l{t)x^i fea,3,i(t)2/gi kz 


i(t) 2 


^^^(r, t) = Qj e' 




(5) 
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FIG. SI: MP-SPIFI system configuration. System schematics in the (a) {x,z) and (b) {y,z) planes, (c) The SPIFI 
modulation mask with Ak = 2/mm. (d) The spatial distribution of diffracted beams in the pupil plane, computed 
using the parameters of the imaging system used in this work (A=1064 nm, NA=0.8, tube lens focal length of 
150 mm). The back aperture diameter was computed using the magnification of the objective lens, Mg, and the 
design tube lens focal length F/=164.5 mm (Zeiss UIS series) with the expression: d = 2 F/NA/Mq. 


where ~ k^ j ]^(t) — k^j i(t), and ki = ‘I'KnxjX is the wavenumber of the illumination beam in the 

mask region. 

To obtain the electric field in the object region, we apply the magnification of the imaging system to the transverse 
wavevectors of the electric field in the mask region. Specifically, the transverse wavevectors in the object region are: 

kx,j,2{t) = —M kx,j,i{t) 

m 


( 6 ) 

( 7 ) 
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so we may write the electric field in the object region as: 


Vj, 2 (^, 1 ) = Oj = ttj exp 


i — Mkxii(t)x 

exp 

i — M kyjp{t) y 

. n-i 


Til 


exp [i fczj. 2 (t) z] 


( 8 ) 


where kz,j, 2 {t) = — [kx,j, 2 {t)f' — [fcyj', 2 (i)]^, and k 2 = 27rn2/A is the wavenumber in the object region. 

The illuminating electric field in the object plane is a sum of the electric fields in the object region: 


N 


£'m(r,t)= ^ 

j=-N 


(9) 


Defining the conjugate of the j**' diffracted beam as Uj^ 2 {^,t) = [uj_ 2 (r, f)]*, we can write the nonlinear intensity as: 


[/in(r,0]'' = 

Finally, the signal from the photodetector is: 

S{t) = 


N N 

Y Vj,2{r,t)uk,2{r,t) 

j^-N k^-N 


( 10 ) 


N N 

Y ^F2(r,<)wfc.2(r,f) 

j——Nk——N 


C{r,t) 


( 11 ) 


B. First-order diffraction only 


While the amplitude modulation masks used in this work are binary, and thus have numerous diffracted orders 
occurring at odd-order harmonics, it is instructive to consider amplitude masks for which only first-order diffraction 
occurs. Formally, this is equivalent to considering an amplitude modulation mask for which the printing process is 
not binary, but instead can faithfully produce sinusoidal amplitude modulation. Using the results of m, the electric 
field for each beam in the object region is: 

uo,2(rU) = ao exp(i/c2z) (12) 

^'l, 2 (I■,^) = aiwi{t) exp[ika:,i, 2 it)x]exp[iky^i^ 2 it)y]ejip[ikz,i, 2 it) z] (13) 

u_n 2 (rU) = aiw-i{t) exp[ika:-i, 2 {t) x]exp[iky-i^ 2 {t) y]exp[ikz-i, 2 it) z] (14) 

where Wjlt) is the temporal window function, which accounts for time-limited diffraction from the modulation mask 
and apodization of the first-order diffracted beams as they scan across the back aperture of the objective lens. For 
a well-aligned system, the window functions are symmetric about time zero, so we can write w{t) = W-i{t) = Wi(t). 
Note that we have assumed that w{t) is a real-valued function, and that the amplitude ai is a real constant. 

The angles in the object region 0i.2(t) and 4'i,2{t) are related to the spatial frequencies in the transverse dimensions. 

From Fig. ^it is clear that the angles of conjugate diffracted beams, e.g., j = ±1, display symmetry about the optic 

axis. Consequently, we can express the lateral and vertical spatial frequencies of the first-order diffracted beams in 
the object region as: 

kx, ±i,2{i) = k2 sin6i±i_2(i) = ±A:2 sin6>i,2(t) (15) 

ky, ±i,2{t) = k2 s\ncj)±i^2{t) = ±^2 sin^i^2(i) (16) 

From these expressions is is clear that k^ _i 2 {t) = —kxp^ 2 {t) and _i 2 (t) = —fcy,i,2(i)- Using these expressions, we 
also find that kzp^ 2 {t) = fcz,-i, 2 (i)- Using these simplifications, the electric field of the j = —1 beam in the object 
region becomes: 


u_y 2 (r, t) = ai w{t) exp [-i kx,i, 2 {t) x] exp [-ifcyp, 2 (i) y] exp [ikz,i, 2 {t) z] 


(17) 
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Thus we can express the total electric field in the object region as: 

1 

Em{r,t) = ^ + 2aiw{t) cos [k^ ^ 2 {t) x + ky ^ 1^2 y] exp[ik^^i^ 2 (t) z] (18) 

i=-i 


Since the illumination beams are tightly focused in the vertical dimension by the objective lens, nonlinear excitation 
results in an optically-sectioned image at the focal plane of the objective lens. In the remainder of this analysis, we 
therefore set z = 0. We also choose to set y — 0 to examine the modulation intensity along the optic axis of the 
system. Finally, we assume that the amplitudes can be set by considering the diffraction efficiency from a perfect 
square grating, such that oq = 1/2 and oi = I/tt. Applying this set of simplifications to the electric field distribution 
above, the illumination intensity for a second-order nonlinear process (77 = 2 ) becomes: 




lb TT^ TT^ 


+ 


1 12 

— w{t)+^w^{t) cos [kx, 2 it) X + Vet] 




cos [ 2 kx, 2 {t) X + 2vc t] + ^ w^it) cos [3kx,2{t) x + 3vc t] 


W^{t) cos [4:kx,2{t) X + 4:Vct] 


(19) 


In the preceding expression, we have explicitly written in the carrier frequency, Vc, which results from the optic 
axis being aligned at a lateral position Xc yf 0, as shown in Fig. The lateral position relative to the center of 

the modulation pattern can be written as x' = Xc + x, where x is the lateral position relative to the optic axis. The 
product of the spatial frequency in the object region and the lateral position x' becomes: 


kx,l,2{t) x' = kxpa^t) Xe + kxp, 2 {t)x = 2tT Vet + kxp, 2 {t) X 


( 20 ) 


The angular spatial frequencies in the object region can be related to the scan time by considering the angles of each 
diffracted beam with respect to the optic axis. We have previously computed these angular frequencies to be |36| : 

kx.iA^) = 1 , 2(0 = 2tt — M fx,i,i(t) = 2 tt — M Akvrt (21) 

ni ni 

The carrier frequency can then be written as Ve = ( 1 x 2 Ini) M Akv^Xe- 

This analysis can be generalized for an arbitrary integer nonlinearity to find: 


2r] 2r) 

[Jm(a:,t)]’' = '^bq{t) cos [qkx, 1 , 2 {t) x + qvet] = '^bq{t) cos[2n q fx,i, 2 {t) x + qvet] (22) 

9=0 9=0 

where the temporal window is absorbed into bq{t). Inserting this expression of the nonlinear illumination intensity 
into the general form of the MP-SPIFI signal in Eq. Q, we find: 

bq(t) cos[2Tr q fxp, 2 (t) X + q Vet] C{x)\ 

\g=o /r 

29 


27] j f V . 277 

= ^ 6g(0 / COS i2TTqM — Akvrtx + qvet \ C[x)\ = ^ Sq{t) 
9=0 ' ^ / / r q^Q 


(23) 


Equation (231 illustrates how 2r] images with varying spatial frequency support are collected in a single rotation of 
the modulation mask. Moreover, single-shot collection of all images is possible because the carrier frequency for each 
image scales with frequency support. Consequently, images can be separated by simply filtering in the modulation 
frequency domain. 


VII. LIMITATIONS OF RESOLUTION ENHANCEMENT 


The plane wave analysis from the previous section indicates that a resolution enhancement of 4x is possible with a 
second-order MP-SPIFI imaging modality. Moreover, the analysis implies that the shape of the OTF for each SPIFI 









13 


order is a top-hat function, meaning that there is no decay in the spatial frequency amplitude throughout the duration 
of the scan. In practice, the circular geometry of the pupil plane imposes limitations on the resolution enhancement 
attainable by apodizing the OTFs for each SPIFI order. This is intuitively understood by observing that the shape 
of each diffracted beam in the pupil plane is highly elliptical. Since each of the diffracted beams scans laterally across 
the X dimension of the pupil, the vertical spatial frequency support of the j**' diffracted order, fyj, is scan-time 
dependent, reducing the throughput energy of the diffracted beams with scan time as well as increasing the vertical 
spread in the line focus created in the object plane. Both of these effects cause a reduction in the peak intensity in 
the object plane, and hence impose strict limitations on the spatial frequency support. 


A. Energy transmission vs. scan time 


We examined the relative energy transmission as a function of scan time by numerically integrating amplitude of 
the diffracted fields described by the pupil function over the pupil aperture: 


Z7T CX 

j{t) = J dcj) J d9 sinO Pj {9,(1), t) 


(24) 


Here Pj{9,(j),t) is the time-dependent pupil function, describing the distribution of the electric field amplitude in the 
pupil plane. The relative energy transmission curves are shown in Fig. ^ Clearly the higher-order diffracted modes 
contribute less to the overall energy of the illumination intensity. 



FIG. S2: Simulated energy transmission of the diffracted orders through the objective lens. Relative energies are 
normalized by the total transmitted energy at scan time t=0, when all diffracted orders are overlaid in the lateral 

dimension. 


B. Vignetting of diffracted illumination beams 

The circular pupil of the objective lens reduces the amplitude of high spatial frequencies in measured data. For 
example, the fourth-order MP-SPIFI image is formed by the second-order nonlinear response of the intensity pattern 
created by interference between the j = ±1 beams: 

S4+{t) = {l4+{r,t)C{r,t))^ = <^[ui,2(r,t)M_i,2(r,t)]^ C{r,t)'^^ (25) 

As these two beams scan off the optic axis of the imaging system, they undergo severe vignetting by the circular pupil 
of the objective lens (Fig. #). causing a loss of transmitted energy and a decrease in the vertical spatial frequency 
support (fy) that is responsible for a further decrease in the peak intensity in the object region. 

We studied the effect of vignetting on lateral spatial frequency support by computing the illumination intensity in 
the object region with the angular spectrum representation iniisi. In order to compute S{t), we must compute the 
illumination intensity in the object region. The vectorial electric field of each diffracted beam in the object region is: 


Vj ,2 (r, t) = Uj,j ,2 (r, t) X -I- Vyj ^2 (r, t)y + j ,2 (r, t) z 


(26) 
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and the illumination intensity becomes: 


N N 

/m(r,t)= Y vy2(r,0 • Ufc,2(r,t) 

k— — N j——N 


( 27 ) 


The measured MP-SPIFI signal is then: 

Sit) = ( 


N N 

Y vy2(r,t) • Ufe,2(r,t) 

k==-N j = -N 


C{r,t) 


(28) 


To obtain the illumination intensity at the focal plane, the Debye integral must be solved numerically for each 
diffracted order: 


>=,1.2 (r,t) 


277 O' 


cos (j) cos 9 cos (0 — 7 ) + sin (j) sin {(j) — 7 ) 


V-*- ^ ^\ r r I L/ \V^ / / oiii. yx 0111 f) 

j_ 2 (r,t) \= d(j) d9 sin 9 Pj [9, sinc/i cos 6 > cos (</) — 7 ) — cos(/> sin (^ — 7 ) 

■ ! J J \ sin0cos(y— 7 ) 


v^,j,2ir,t) 


X\l— cos 9 exp (i k 2 z cos 9) exp [i k 2 p sin 9 cos (^ — v?)] 


In cartesian coordinates, the pupil function has the form of a two-dimensional Gaussian distribution: 


Pjixp,yp,t) = Qj exp < - 


-xojit) 


Wx 


exp • 


Vp - yodit) 


Wy 


(29) 


(30) 


where {xp,yp) are coordinates in the pupil plane, XQj{t) and yopit) are the lateral and vertical shifts of the 
diffracted order in the pupil plane, and Wx and Wy are the widths of the Gaussian distributions. To solve the integral 
equation in Eq. (291, the pupil function must be converted to spherical polar coordinates, iFo,9,(j>). Note that the 
radial coordinate is equivalent to the focal length of the objective lens because the pupil function is to be described 
on the surface of an aplanatic focal sphere |35]. Applying the coordinate change, we find: 


Xp = Fq sin 9 cos (j) 
yp = Fq sin 9 sin (j) 


and the pupil function in Eq. (301 becomes: 


Pj {9, (j), t) = Qj exp < — 


Fo sin 9 cos (j) — xqj {t) 


Wx 


exp < - 


Fo sin 9 sin (j) — yop (t) 


Wy 


(31) 

(32) 


(33) 


The lateral shifts of the diffracted orders in the pupil plane are determined by the spatial frequency of the modulation 
mask as a function of scan time, the wavelength of the illumination light, and the focal length of the tube lens: 


xo,jit) = Ft j — fx,ii'^) = Ft j — Akvrt = j xo,i it) 

Til Til 

yoj = Ftj — fy,i= Ftj — = J 2 /0.1 

Tl\ 


(34) 

(35) 


The widths of the distributions are set by the focal lengths of the cylindrical and tube lenses, as well as the input 
beam size. Win, and the illumination wavelength: 


Wx 

Wy 


XFt 

TTWin 

Ft 


(36) 

(37) 


where Fc is the focal length of the cylindrical lens. 

We computed the illumination intensity in the transverse plane located in the focus of the illumination objective 
(z = 0) for several scan times, considering only the j = 0 and j = ±1 beams. From Fig. f] it is clear that as the 




















15 


first-order diffracted beams scan away from the optic axis in the lateral {x) dimension, the vertical spatial frequency 
support decreases and causes a corresponding broadening of the illumination beams in the vertical dimension. 

Finally, the vector-focusing formalism was used to compute the OTF of each MP-SPIFI order using the parameters 
of the microscope described in the Methods (A=1064 nm; A/c=70/mm; 150 mm focal length tube lens; 50x/0.8 NA 
Zeiss air-immersion UIS series objective lens). To compute the MP-SPIFI signal, we assumed a point emitter located 
at the focal point of the objective lens, rp = (0,0,0). The point emitter was modeled by a Dirac-J distribution, 
(5(r — rg). Since the contrast distribution we consider is a delta function, we need only compute the intensity at rg to 
obtain the MP-SPIFI signal: 


Sit) = ^[Aii(r, t)f 5(r - rg)^^ = [/in(rg, t)f (38) 

For the simulated TP-SPIFI data presented in Fig.|^ we considered a second-order nonlinearity (ry = 2), and included 
only first-order diffraction from the modulation mask, where og = 1/2 and oi = I/tt were set by the diffraction 
efficiency of a square grating. 
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FIG. S3: Intensities in the object and pupil planes, computed for several scan times with the angular spectrum 

representation. 









































